Image display method, program, and image display apparatus

ABSTRACT

A table is allocated indexes corresponding to display positions on a monitor screen, respectively, and data indicating an unexposed part at each of the display positions is stored in each index of the table. Focal position information is acquired from Exif header information on image data of an image, and a location of one of the indexes in the table at which no image data is registered is searched in an ascending order of the indexes. According to the table, the images are arranged so that the focal position of each image differs from the unexposed part at the display position of the image.

RELATED APPLICATIONS

This application is a division of co-pending U.S. patent application No.10/869,919, filed on Jun. 18, 2004, (now allowed), which is incorporatedby reference herein in its entirety, as if fully set forth herein, andclaims the benefit of priority under 35 U.S.C. §119, based on JapanesePriority Application Nos. JP 2003-176856, filed Jun. 20, 2003, and JP2004-048430, filed Feb. 24, 2004, which are incorporated by referenceherein in their entirety, as if fully set forth herein.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image display method for displayingdigital image data on a television monitor screen.

2. Description of the Related Art

Following start of digital high-vision broadcasting, televisionreceivers for digital high-vision have spread widely, thus enablingusers to enjoy higher-quality and more impressive videos at home thanbefore. Further, electronic still cameras have spread, so that usershave increasingly displayed and enjoyed electronic still images andmoving images not only on screens of personal computers but alsotelevision monitor screens.

To view still images or moving images using a television receiver, auser normally looks at a list of thumbnail images displayed on atelevision monitor screen, selects one of the thumbnail images, andthereby selects data to be reproduced.

Meanwhile, memories with rapidly increased storage capacities haverecently been available. The number of images to be stored is increasingby leaps and bounds, accordingly. A memory card, for example, can storeover 500 digital photographs.

It has been difficult for the user to find desired data from among imagedata in large quantities.

Examples of a technique for efficiently arranging large quantities ofdata objects on the monitor screen, and for enabling the user himself orherself to easily select one of the objects with user's own eyes includea technique for moving thumbnail images while overlapping them, and forfacilitating finding desired data as disclosed in Japanese PatentApplication Laid-Open (JP-A) No. 2001-306375. They also include atechnique, disclosed in JP-A No. 9-259130, for displaying image data ina three-dimensional space, moving a virtual eye view, and therebyfinding desired data.

The conventional techniques have, however, the following disadvantages.Since the thumbnail images are displayed while being overlapped with theother thumbnail images, important parts for discrimination ofphotographs are often hidden. A photograph A and a photograph B arepresent, for example, as shown in FIGS. 14A and 14B. In the photographA, children who are “running in a race” on the left of the image is animportant part. In the photograph B, a person on the right of the imageis an important part. If these photographs A and B are displayed whilebeing overlapped with the other photographs, the photographs A and B aredisplayed with their important parts hidden as shown in FIG. 14A.

To prevent them, as proposed in the JP-A Nos. 2001-306375 and 9-259130stated above, there is no other choice but for the user to carry outoperations such as scrawling of the displayed image or moving a positionof a virtual eye view so as to be able to view the important part ofeach photograph, and then to discriminate the photograph.

The present invention has been achieved in light of the conventionaldisadvantages. It is an object of the present invention to provide animage display method which enables a user to easily find a desired imagewhile large quantities of images are efficiently arranged on a monitorscreen.

Further, each of almost all photographed digital images can be dividedinto a region which a photographer regards as a subject and the otherregion. The former region will be referred hereafter to as “region ofinterest” and the latter region will be referred to as “region of nointerest”.

If an operator of an image display apparatus is to search a desiredimage from large quantities of digital images, it is considered that theoperator determines the desired image based on the region of interest ineach of the digital images.

Conventionally, to create a thumbnail image, image data is uniformlyreduced without discriminating the region of interest from the region ofno interest. Due to this, when a plurality of images are displayed as alist of thumbnail images, the regions of interest in the respectiveimages are displayed in small sizes. As a result, it isdisadvantageously difficult to find a desired image based on the regionsof interest as criteria of determination while looking down on the largequantities of digital images.

Considering the disadvantage, a method for detecting a face area in animage, enlarging the detected face area, and displaying a thumbnailimage in a list is proposed, as disclosed in JP-A No. 2002-358522.According to this method, a system having an improved performance forsearching a desired digital image can be realized by detecting the facearea as the region of interest, enlarging the detected face area, anddisplaying the enlarged region.

With this method, since the list of thumbnail images is displayed withthe respective regions of interest enlarged, the number of images whichcan be displayed once on the monitor screen is fixed in advance. Due tothis, if the number of images exceeding the fixed number of images whichcan be displayed once on the monitor screen, such functions as ascrawling function and a function of turning over pages are additionallyrequired, thereby disadvantageously complicating user's operation.

In addition, if both enlarged regions of interest and original imagesare displayed as thumbnail images, the number of images which can bedisplayed once on the monitor screen is disadvantageously, furtherreduced.

If the face area cannot be detected from a target image, no measures canbe disadvantageously taken.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, there is provided animage display method comprising a step of simultaneously displaying aplurality of images on a screen while overlapping at least two imageswith each other, the method comprising steps of: acquiring relatedinformation associated with the at least two images; and determiningwhich of the at least two images is arranged at an upper position whenoverlapping the at least two images based on the acquired relatedinformation.

According to another aspect of the present invention, there is providedan image display method comprising a step of simultaneously displaying aplurality of images on a screen while overlapping at least two imageswith each other, the method comprising steps of: acquiring relatedinformation associated with the at least two images; and determiningboth of or one of an exposure amount and an exposure position of theother one of the at least two images, which has a hidden part and anexposed part as a result of overlapping one of the at least two imageswith the other image, based on the acquired related information.

According to yet another aspect of the present invention, there isprovided an image display method comprising a step of simultaneouslydisplaying a plurality of images on a screen while overlapping at leasttwo images with each other, the method comprising steps of: detectingface areas of the at least two images, respectively; and determiningwhich of the at least two images is arranged at an upper position whenoverlapping the at least two images based on magnitudes of therespective face areas.

According to still another aspect of the present invention, there isprovided an image display method comprising a step of simultaneouslydisplaying a plurality of images on a screen while overlapping at leasttwo images with each other, the method comprising steps of: detectingface areas of the at least two images, respectively; and determiningboth of or one of an exposure amount and an exposure position of theother one of the at least two images, which has a hidden part and anexposed part as a result of overlapping one of the at least two imageswith the other image, based on positions of the respective face areas.

According to the image display apparatus and the image display method ofthe present invention, if a plurality of images are displayed as a listwhile overlapping them with one another, parts considered to be regionsof interest of the respective images are detected, thereby controllingimage display so as not to overlap the regions of interest with oneanother. As a result, all the regions of interest of the images can bealways exposed, whereby the constitution capable of realizing bothimproved user image search performance and display of a large quantityof images can be provided.

If a plurality of images are displayed as a list while overlapping themwith one another, the images are controlled to be sequentiallysuperposed in a descending order of magnitudes of the regions ofinterest of the images. As a result, by displaying a large quantity ofdigital images photographed by a user using a digital camera on themonitor screen while overlapping them with one another, the images canbe efficiently arranged as the list thereon with a high exposure ratioof each region of interest. In addition, even if the images are arrangedefficiently, the images are displayed so as not to overlap importantparts for user's discrimination of photographs with one another, basedon information characteristic of the individual images.

Namely, according to the present invention, the user can easily find adesired photograph.

Furthermore, if a plurality of images are displayed as the list whileoverlapping them with one another, the parts considered to the regionsof interest of the respective images are detected, thereby controllingimage display so as not to overlap the regions of interest with oneanother. As a result, all the regions of interest of the images can bealways exposed, whereby the constitution capable of realizing bothimproved user image search performance and display of a large quantityof images can be provided.

Moreover, if a plurality of images are displayed as the list whileoverlapping them with one another, the images are controlled to besequentially superposed in a descending order of magnitudes of theregions of interest of the images. As a result, the images can beefficiently arranged as the list thereon with a high exposure ratio ofeach region of interest.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram which depicts configuration of a televisionreceiver according to a first embodiment of the present invention;

FIG. 2 is a block diagram of a remote controller of the televisionreceiver according to the first embodiment;

FIG. 3 is a flowchart which depict an operation flow of a displaycontrol unit 110 according to the first embodiment;

FIG. 4 depicts layout information included in an information tableemployed for overlap control exercised by the display control unit 110according to the first embodiment;

FIG. 5 depicts contents of the information table employed for theoverlap control exercised by the display control unit 110 according tothe first embodiment;

FIGS. 6A and 6B depict functions of the overlap control according to thefirst embodiment;

FIG. 7 is a flow chart which depicts an operation flow of a displaycontrol unit 110 according to a second embodiment of the presentinvention;

FIGS. 8A to 8C depict types of an exposure control mask employed foroverlap control exercised by the display control unit 110 according tothe second embodiment;

FIG. 9 is a flow chart which depicts an operation flow of the displaycontrol unit 110 according to the second embodiment;

FIGS. 10A to 10F depict operation examples of the overlap controlaccording to the second embodiment;

FIGS. 11A to 11F depict functions of the overlap control in an exampleof overlap display without using the present invention and thoseaccording to the second embodiment of the present invention;

FIG. 12 is a flow chart which depicts an operation flow of a displaycontrol unit 110 according to a third embodiment of the presentinvention;

FIGS. 13A to 13C depict operation examples of overlap control accordingto the third embodiment;

FIGS. 14A and 14B depict a display example of an image list withoutusing the present invention and that using the present invention,respectively;

FIGS. 15A and 15B depict examples of an image display on a liquidcrystal monitor screen of a digital camera employed in the embodimentsof the present invention;

FIGS. 16A and 16B depict a state of a liquid crystal monitor screenduring photographing using a digital camera which is employed in thefirst and the second embodiment of the present invention and which canrecord a focal position;

FIG. 17 is a block diagram which depicts configuration of an imagedisplay apparatus according to the embodiments of the present invention;

FIG. 18 is a flow chart which depicts a processing flow for a facedetection processing used as one method for detecting a region ofinterest;

FIG. 19A depicts one example of image data, and FIG. 19B depicts oneexample of a determination result of a face area after the facedetection processing;

FIG. 20A depicts one example of image data, and FIG. 20B depicts oneexample of a region-of-interest determination processing based onpicture-taking information;

FIG. 21 is a flowchart which depicts a processing flow for aregion-of-interest determination processing according to a fourthembodiment of the present invention;

FIG. 22A depicts one example of displaying overlapped images at random,and FIG. 22B depicts one example of displaying overlapped images whileconsidering magnitudes of respective regions of interest;

FIG. 23 is a flowchart which depicts a processing flow for an overlapdisplay processing;

FIG. 24 depicts one example of displaying an image list while images areoverlapped;

FIG. 25 is a flowchart which depicts a processing flow for aregion-of-interest determination processing according to a fifthembodiment;

FIG. 26 depicts a result of detecting a region of interest according toa sixth embodiment; and

FIG. 27 is a flowchart which depicts an operation flow of a displaycontrol unit 110 according to a seventh embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will be described hereinafter withreference to the drawings.

(First Embodiment)

A first embodiment of the present invention will be described. FIG. 1depicts configuration of a digital television receiver 100 as an imagedisplay apparatus to which the present invention is applied.

In FIG. 1, a signal received at an antenna, not shown, is input to atuner unit 101. The tuner unit 101 carries out processings such asdemodulation and error correction to the input signal, and generatesdigital data in a format called “transport stream (TS)”. The tuner unit101 descrambles the generated TS data, and outputs the descrambled TSdata to a demultiplexer unit 102.

The demultiplexer unit 102 fetches video data and voice data from the TSdata in which video and voice data on a plurality of channels,electronic program guide (“EPG”) data, digital broadcasting data and thelike are time-division multiplexed, and which is input from the tunerunit 101, and outputs the fetched video data and voice data to a videoand voice decoder unit 103. The video data processed by the video andvoice decoder unit 103 is written into a video memory 107, and output toa display through a display synthesization unit 108. The voice dataprocessed by the video and voice decoder unit 103 is output to a voiceoutput unit 104.

In FIG. 1, reference symbol 109 denotes a card slot, to which aremovable memory card or the like is connected. In the first embodiment,the card slot 109 is used to exchange data between a digital televisionreceiver 100 and a digital camera. Reference symbol 105 denotes an imagedata input unit, which reads digital image data from the memory card orthe like connected to the card slot 109. Reference symbol 106 denotes animage processing unit, which carries out an image processing to thedigital image data input from the image data input unit 105, and whichwrites the resultant digital image data into the video memory 107.

Reference symbol 110 denotes a display control unit. The display controlunit 110 instructs the image data input unit 105 to read the digitaldata image, the image processing unit 106 to execute the imageprocessing, the video memory 107 to write the display image data, thedisplay synthesization unit 108 to perform a synthesization processing,and a storage unit 113 to store information. The display control unit110 also acquires input data from a remote controller 114 through a userinput unit 111, and acquires photographic data allocated to the digitalimage data through the image data input unit 105. In this embodiment,the display control unit 110 corresponds to an overlap determinationunit and an exposure determination unit.

FIG. 2 depicts one example of the remote controller 114. It is notedthat FIG. 2 depicts only buttons for performing operations for realizingfunctions necessary to describe this embodiment. Therefore, buttonsnecessary for an actual receiver are not limited to those shown in FIG.2.

In FIG. 2, reference symbol 201 denotes a light emitting unit forholding an infrared communication between the remote controller 114 andthe user input unit 111, serving as a light receiving unit, shown inFIG. 1. Reference symbol 202 denotes a power key for turning on and offa power, and 203 denote a cursor and decision key having a decisionbutton arranged in the middle of upper, lower, left, and right buttons.Reference symbols 204 and 205 denote an up and down key composed of twobuttons, i.e., an Up button and a Down button, 206 denotes a numeric keyhaving numeric buttons 1 to 9 and 0 arranged in a matrix, and 207denotes a color key having four color buttons of blue, red, green, andyellow aligned.

Further, reference symbol 208 denotes a monitor mode key for switchingover a display mode of a monitor of the television receiver, and 209denotes a “viewer” key for calling an image viewer function.

The digital image used in this embodiment is still image data or movingimage data photographed by the digital camera. As the still image data,data subjected to a JPEG compression processing by the digital camera,and recorded in the memory card as a still image data file is used.Likewise, as the moving image data, data subjected to the JPEGcompression processing for each frame to provide images, and recorded inthe memory card as a moving image data file is used. It is assumed thatthe “image file” denotes either the still image data or the moving imagedata in the description given hereafter.

Each image file is allocated information during photographing using thedigital camera as information associated with the image. Examples of theallocated information during photographing include photographic scenemode information, focal position information that indicates a focalposition in a finder, strobe light emission state information,information that indicates a distance between the camera and a subject,and zoom state information as well as a photographic date and a cameratype.

For the digital camera employed in this embodiment, one of “left”,“middle”, and “right” is recorded as the focal position information inthe finder during photographing.

FIGS. 15A and 15B depict examples of image display on a liquid crystalmonitor screen of the digital camera employed in this embodiment. FIG.15A depicts a state before a user touches a shutter button, and FIG. 15Bdepicts a state, after the user lightly touches the shutter button, inwhich the digital camera brings a rectangular region on the left intofocus.

(Processing Operations)

In the digital television receiver 100 according to this embodiment,display of a list of digital images is initiated when the user insertsthe memory card into the card slot 109, and calls the image viewerfunction by operation of the remote controller 114.

FIG. 3 is a flowchart which depicts part of operations of the imageviewer function in the digital television receiver 100 centering aroundoperations performed by the display control unit 110. FIG. 5 depictstable data used when the display control unit 110 exercises layoutcontrol for the display of the image data list. FIG. 4 depicts oneexample of image display positions corresponding to respective indexesin FIG. 5. The processing operations of the display control unit 110will be described with reference to FIGS. 3 to 5.

When the user depresses the “viewer” key 209 on the remote controller114 shown in FIG. 2, the display control unit 110 starts processingsshown in FIG. 3.

First, the display control unit 110 reads and stores the total number ofall image files stored in the memory card inserted into the card slot109 through the image data input unit 105 (in a step S301). The displaycontrol unit 110 reads Exif header information on a first image file (insteps S302 and S303).

The display control unit 110 reads a content of a first index of thetable shown in FIG. 5 (in a step S304). If no image file is registeredin the first index (“Yes” in a step S305), the display control unit 110compares focal position information included in the read Exif headerinformation with “an unexposed part” in the first index of the table (ina step S307). If they coincide (“Yes” in a step S308), the displaycontrol unit 110 reads a content of a second index of the table (insteps S306 and 304). If no image file is registered in the second index(“Yes” in the step S305), the display control unit 110 compares thefocal position information included in the read Exif header informationwith “an unexposed part” in the second index of the table (in the stepS307). If they do not coincide (“No” in the step S308), the displaycontrol unit 110 registers the first image file which is now beingprocessed in this index of the table (in a step S309).

In FIG. 5, an image file named “IMG001.JPG” having the focal positioninformation of “left” is not registered in the first index since theunexposed part of the first index is “left”, which is equal to the focalposition information of IMG001.JPG. The image file IMG001.JPG isregistered in the second index of the table since the unexposed part inthe second index is right, which differs from the focal positioninformation “left” on the image file IMG001.JPG.

In this way, the display control unit 110 searches for an empty positionin an ascending order of index of the table. If the display control unit110 finds an empty position, the display control unit 110 compares theunexposed part corresponding to the index with the focal positioninformation on the image file. If they coincide, the display controlunit 110 carries out the same processing to the next index of the table.

Likewise, the display control unit 110 continues to carry outprocessings to second, third, and the following image files stored inthe memory card. When completing with the processing to an M^(th) imagefile (where M is the total number of image files), the display controlunit 110 carries out a display processing to each image file accordingto contents registered in the tables. The display control unit 110 readsdata on each of the image files from the memory card through the imageinput unit 105 (in a step S314), feeds the read data to the imageprocessing unit 106 (in a step S318). The image processing unit 106decodes the data (in a step S318) and writes the display data into thevideo memory 107, whereby the image is displayed (in a step S319).

If all the image files are displayed by the processings shown in FIG. 3,then the user operates the upper, lower, left, and right buttons of thecursor and decision key 203 on the remote controller 114, moves acursor, and selects a desired image file. If the user depresses thedecision button while the cursor is pointing out the desired image file,only the selected image file can be displayed on the monitor screen. Itis noted that these processings are not shown in the figures.

(Functions and Advantages)

Functions obtained as a result of the processings performed by thedisplay control unit 110 described above will be described. FIG. 6Adepicts a state in which the image file IMG001.jpg having the focalposition information of “left” is arranged by the processings above notat the position “1” at the position “2” (note that encircled numbers inFIGS. 4, 5, 6A and 6B, 10A to 10F, 11A to 11F, and 13A to 13C arereplaced by bracketed numbers). As a result, the subject part (region ofinterest) on the left of this image file is exposed. As compared with aninstance of arranging the image file IMG001.jpg at the position “1” asshown in FIG. 14B, the user can easily find this photographic imagefile.

Namely, according to the first embodiment of the present invention,based on the information characteristic of each photograph, a partsuited for determination of the photograph can be exposed. In thisembodiment, the focal position information recorded during photographingis used for the determination. Alternatively, if the digital camera canset a mode, in which the subject position or the focal position isfixed, as the photographic scene mode, information on this photographicscene mode can be used. If the digital camera having photographic scenemodes of, for example, “person on the right” and “person on the left” asshown in FIG. 16 is used to photograph an image, and Exif headerinformation on this image file includes information of “person on theright”, then the position of the image file displayed in the image filelist is controlled so as to expose a right part of the image. Thus, thefunctions stated above can be attained.

In the first embodiment, the instance of displaying only 12 photographicimage files once on the monitor screen has been described for brevity ofdescription. The number of image files which can be displayed once onthe monitor screen may be set larger than 12.

(Second Embodiment)

A second embodiment of the present invention will be described. Since adigital television receiver employed in this embodiment is equal inconfiguration to that shown in FIG. 1, it will not be described herein.

In the second embodiment, digital image files photographed using adigital camera similar to employed in the first embodiment is used.Differently from the first embodiment, however, the digital camera whichparticularly records information on a photographic date, information asto a direction of the camera during photographing (that is, informationthat indicates a landscape photograph or a portrait photograph), and amode setting of “auto mode”, “landscape mode”, “portrait mode”, or“night mode” as a photographic scene mode during photographing. In thisembodiment, information on the setting of the photographic scene modecorresponds to information on the photographic mode.

(Processing Operations)

In the digital television receiver 100 according to this embodiment,display of a list of digital image files is initiated when the userinserts the memory card into the card slot 109, and calls the imageviewer function by operation of the remote controller 114.

FIG. 7 is a flowchart which depicts part of operations of the imageviewer function in the digital television receiver 100 centering aroundoperations performed by the display control unit 110. FIGS. 8A to 8Cdepict types of an exposure control mask employed for processings shownin FIG. 7. Exposure control masks different in exposure amount andexposure position are prepared according to focal positions orphotographic scene modes. The processing operations of the displaycontrol unit 110 will be described with reference to FIGS. 7 and 8A to8C.

The display control unit 110 acquires the total number of image filesrecorded in the memory card (in a step S701), and reads Exif headerinformation on a first image file (in a step S702).

If focal position information is included in the Exif header informationon the first image data (“No” in a step S704), the display control unit110 selects an exposure control mask having a focal positioncorresponding to the focal position information from among three masksshown in FIG. 8A, i.e., a left exposure mask, a center exposure mask,and a right exposure mask (in a step S705). If no focal positioninformation is included in the Exif header information on the image file(“Yes” in the step S704) but photographic scene mode information isincluded therein (“No” in a step S706), the display control unit 110selects an exposure control mask corresponding to the photographic scenemode from among those shown in FIG. 8B (in a step S707). If neither thefocal position information nor the photographic scene mode informationare included in the Exif header information on the image file, thedisplay control unit 110 selects a standard exposure mask shown in FIG.8C (in a step S708).

For subsequent image files, the display control unit 110 carries out thesame processings to determine the exposure control masks for therespective image data (in steps S709, S710, S703, and the following).

FIG. 9 is a flow chart which depicts a processing flow for a displayprocessing performed by the display control unit 110. FIGS. 10A to 10Fdepict the image data used in this embodiment and states of the imagefiles on the monitor screen under the display control shown in FIG. 9.FIG. 10A depicts the image file used in this embodiment. In FIGS. 10A to10F, the image files used herein and denoted by “1” to “10” alternatelyhaving focal position information of “left”, “center”, and “right” arearranged in this order.

The display processing will be described with reference to FIG. 9 andFIGS. 10A to 10F.

The display control unit 110 reads a thumbnail image of the first imagefile from the memory card (in steps S901 and S902). The image processingunit 106 decodes the thumbnail image (in a step S903), and carries out amask processing to the decoded image file using the exposure controlmask selected in the processing shown in FIG. 7 (in a step S904). Asthis mask processing, a processing for burying flag data that indicateswhether each pixel data is an exposed part or an unexposed part iscarried out.

In this embodiment, a lower one bit of a blue component pixel (0 to 255bits) is used as the flag data. That is, a blue component pixel havingthe lower one bit of “0” means the unexposed part, and a blue componentpixel having the lower one bit of “1” means the exposed part.

Next, the display control unit 110 arranges the first image file whichis now being processed at a preset standard position on the monitorscreen (in a step S905). In this embodiment, the reference location isset at a left end of a region which is to be displayed as the list onthe monitor screen. The display control unit 110 determines whether thepixel including the flag data that indicates the exposed part is presentunder the region in which the image file is to be arranged (in a stepS906). Since no other image file is present for the first image file,the first image file is arranged at the standard position as it is.

If the flag data that indicates an exposed position is present under thefirst image file already arranged, that is, if the second image file isoverlapped with the exposed part of the other image file while thedisplay control unit 110 processes the second file similarly to thefirst image file (in steps S907 and S908), the display control unit 110moves a position of the arranged second image file in a right directionby as much as certain number of pixels, and thereby rearranges thesecond image file (in a step S909).

FIG. 10B depicts a state in which the display processing up to thesecond image file is finished. Specifically, the second image file isinitially arranged at the standard position by the arrangementprocessing. However, since the second image file is overlapped with theexposed part of the first image file, the position of the second imagefile is moved in the right direction. When there is no overlap of thesecond image file with the exposed part of the first image file, thearrangement processing is finished.

The third image file is then processed. During this processing, thethird image file is initially arranged at the standard position, andoverlapped with the exposed part of the first image file. A position ofthe third image file is moved, and the third image file is therebyrearranged. As can be seen from FIG. 10A, however, the third image fileis a photograph having the focal position information of “right” in thisembodiment. Therefore, for the third image file, the display controlunit 110 selects the right exposure mask in the processing shown in FIG.7. In this case, therefore, the display control unit 110 moves the thirdimage file in a left direction relative to the standard position (in astep S910) to a position at which the third image file is not overlappedwith the exposed part of the other image file, and thereby rearrangesthe third image file. At this time, the display control unit 110 slidesthe already arranged image files (the first and the second image filesin this embodiment) rightward so that the third image file does notprotrude from the list display region surrounded by a dotted line. Inaddition, the display control unit 110 moves the standard positionrightward by as much as the same number of pixels as that for the firstand the second image files (in a step S913). FIG. 10C depicts thisstate.

The fourth image file and the following are similarly processed, and thearrangement processing is proceeded as shown in FIGS. 10D to 10F.

(Functions and Advantages)

Functions obtained as a result of the processings performed by thedisplay control unit 110 described above will be described. Beforestarting the description, examples of display to which the presentinvention is not applied will be described with reference to FIGS. 11Ato 11F. FIG. 11A depicts a display example of displaying the image fileswith the image files simply overlapped with one another without applyingthe present invention. In the example of FIG. 11A, if focal positions ofall the image files are at the center of the respective image files asshown in FIG. 11B, substantially all of the focal positions are exposedas shown in FIG. 11C. However, as shown in FIG. 11D, if image filesincluding the image files the focal positions of which are not at thecenter are displayed as the list, exposed parts of some of the imagefiles are greatly hidden as shown in FIG. 11E.

If the user is to photograph images having a plurality of focalpositions using the digital camera as employed in this embodiment, theuser normally performs photographing while confirming that the camerabrings a part that the user considers important into focus. That is, ifan image file is displayed with a region near its focal positionshidden, this often means that the subject which the user considersimportant, i.e., the important part for determination of the image fileis hidden.

If the display control unit 110 according to this embodiment performsthe display processings as stated above, by contrast, all image filescan be displayed without causing regions near their respective focalpositions to be hidden.

Furthermore, although not shown in FIGS. 11A to 11F, even if no focalposition information is present for an image file, the exposed part andthe exposed amount can be controlled according to the photographic scenemode of the image file by switching the exposure control mask in thephotographic scene mode such as “landscape mode” or “portrait mode” inthe exposure control mask selection processing described above withreference to FIGS. 7 and 8A to 8C. For example, if the image file is alandscape photograph, the entire landscape often remains as “impression”for the user. Due to this, in order to discriminate and select the imagefile, it is necessary to increase the exposed part of the image file. Ifthe image file is a portrait photograph, then it often suffices toexpose the image file so as to discriminate a face, and it isunnecessary to increase the exposed part of the image file.

By thus exercising control while reflecting the photographic scene modeof each image file in the control, it is possible to efficiently arrangethe image files in accordance with characteristics of the respectiveimage files while maintaining easiness of discrimination.

In the second embodiment, only an instance of arranging the image fileslaterally to be aligned while overlapping the image files has beendescribed as a method for displaying a list of images. However, in lightof the concept of the present invention, it is natural that the imagedisplay method is not limited to that described in this embodiment. Thepresent invention can be also applied to an instance of displaying theimage files longitudinally and an instance of displaying the image filesboth laterally and longitudinally.

In this embodiment, the use of the lower one bit of the blue componentpixel as the flag data in the mask processing using the exposure controlmask has been described. However, in light of the concept of the presentinvention, it is natural that the flag data is not limited to thatdescribed in this embodiment. The present invention can be also appliedto an instance of providing a dedicated flag bit for each pixel and aninstance of using bits of data that indicate an alpha blending ratio.

In this embodiment, only the two types of photographic scene modes of“portrait mode” and “landscape mode” have been taken as an example.Needless to say, the photographic scene modes according to the presentinvention are not limited to them. “Night scene mode” and “macro scenemode” may be similarly available.

(Third Embodiment)

In the second embodiment, the image files are arranged so as to beefficient and easy to discriminate based on the Exif header informationsuch as the focal position information and the photographic scene modeinformation. In light of the gist of the present invention, the Exifinformation used for the arrangement is not limited to these pieces ofinformation. The image files can be efficiently arranged based on, forexample, the photographic date information.

Since a digital television receiver employed in this embodiment is equalin configuration to that shown in FIG. 1, it will not be describedherein.

(Processing Operations)

FIG. 12 is a flowchart which depicts part of operations of the imageviewer function in the digital television receiver 100 centering aroundoperations performed by the display control unit 110. FIGS. 13A to 13Cdepict image files used in this embodiment. Image files are arranged attime series, and photographic time is shown below each image file.

The display control unit 110 reads Exif header information on firstimage file from the memory card (in steps S1001 to S1002). The displaycontrol unit 110 further reads a thumbnail image of the first image file(in a step S1003). The display control unit 110 feeds the read imagefile to the image processing unit 106, and the image processing unit 106decodes the image file (in a step S1004).

Next, the display control unit 110 reads a photographic time of(N−1)^(th) image file (in a step S1005). If a difference in photographictime between the N^(th) image file and the (N−1)^(th) image file iswithin ten seconds (files at the positions of “5” or “6” in FIGS. 13A to13C), then the display control unit 110 controls image display so as toset a larger overlapped amount between the N^(th) image file and the(N−1)^(th) image file (in steps S1006 and S1007). If the difference inphotographic time between the N^(th) image file and the (N−1)^(th) imagefile exceeds ten seconds, the display control unit 110 controls theimage display so as to set the overlapped amount therebetween at apredetermined ordinary amount (in steps S1006 and S1008).

Likewise, the same processings are carried out to (N+1)^(th) image fileand the following. When the processings are carried out to all imagefiles recorded in the memory card, the processings are finished (insteps S1009 and S1010).

By thus controlling the image display, the image files are displayed asa list as shown in FIG. 13C.

(Functions and Advantages)

As compared with an instance in which all the image files are overlappedwith one another with the same overlapped amount by the processingsstated above (FIG. 13B), more image files can be arranged once on themonitor screen according to this embodiment. If the photographic timedifference is as small. as ten seconds or less, only one image file(file at the position “6” in FIG. 13C) among the image files is exposedmore than the other image files (files at the positions “4” and “5” inFIG. 13C). Since the exposed parts are reduced, it is consideredundesirable for discrimination of photographs. However, for the imagefiles the photographic time difference among which is small as seen inthis example, the user can estimate contents of the less exposed imagefiles (files at the positions “4” and “5”) from the more exposed imagefile (file at the position “6”). Normally, therefore, this is not ahindrance for image file selection.

In other words, the processings performed by the display control unit110 according to this embodiment enables efficient display of a list ofimage files in accordance with the difference in photographic time amongthe image files.

(Fourth Embodiment)

FIG. 17 is a block diagram which depicts configuration of an imagedisplay apparatus according to a fourth embodiment.

In FIG. 17, reference symbol 1101 denotes the image display apparatus.Reference symbol 1201 denotes a control CPU which controls entirety ofthe image display apparatus 1101. Reference symbol 1203 denotes an imagedisplay unit, which is composed of a device which can realize a thinfilm display such as a liquid crystal display or a plasma display. Theimage display unit 1203 displays an image file indicated by the controlCPU 1201. In this embodiment, the control CPU 1201 corresponds to anoverlap determination unit and an exposure determination unit, and theimage display unit 1203 corresponds to a display.

The image display apparatus 1101 also display a list of a plurality ofimage files while overlapping them with one another, which is a featureof the present invention.

Reference symbol 1205 denotes an image storage unit, which is composedof a nonvolatile memory such as a hard disk. The image storage unit 1205can accumulate and store a large quantity of images photographed by adigital camera or a digital video camera, and fetched by the imagedisplay apparatus 1101.

Reference symbol 1207 denotes a memory card. The memory card 1207 isdetachable. The memory card 1207 can be also connected to an imagephotographing device such as the digital camera or the digital videocamera. Photographed image files can be fetched into the image displayapparatus 1101 via this memory card 1207.

Reference symbol 1209 denotes an external input terminal. Via thisterminal 1209, the image files photographed by the image photographingdevice such as the digital camera or the digital video camera can bedirectly fetched into the image display apparatus 1101.

Reference symbol 1211 denotes an image decoder unit. A format of theimage file stored in the image storage unit 1205 or the memory card 1207or that fetched from the external input terminal 1209 is converted intoa data format which can be handled by the constituent elements of theimage display apparatus 1101 such as the control CPU 1201.

Reference symbol 1213 denotes a face detection processing unit. The facedetection processing unit 1213 determines whether a person image ispresent in the fetched image file. If the person image is present, theface detection processing unit 1213 carries out a processing forspecifying a region of the person image. Details of the processing willbe described later.

Reference symbol 1215 denotes a cache memory. The cache memory 1215 isused to, for example, temporarily store the image file decoded by theimage decoder unit 1211, to store the image file displayed by the imagedisplay unit 1203, and to temporarily store the image file for the facedetection processing unit 1213.

FIG. 18 is a flowchart which depicts a processing flow for a facedetection processing used as one method for detecting a region ofinterest.

Before executing display of a list of image files while overlapping themwith one another, the face detection processing unit 1213 executes theface detection processing to display target image files. When theprocessing starts, a step S1101 is executed.

In the step S1101, the face detection processing unit 1213 executes aprocessing for finding a region which includes skin color data fromcolor data included in the image file. Thereafter, a step S1103 isexecuted.

In the step S1103, the face detection processing unit 1213 executes apattern matching processing for determining whether the target imagefile matches to a pattern indicating features of a face, i.e., imagepatterns of eyes and mouth. Thereafter, a step S1105 is executed.

At the step S1105, based on processing results of the steps S1101 andS1103, i.e., based on whether a face area is present in the target imagefile, the processing is branched. Namely, if the face area is present,the processing goes to a step S1107. If the face area is not present,the processing is finished.

In the step S1107, the face detection processing unit 1213 writesinformation of the area determined as the face area by the processingresults of the steps S1101 and S1103 into the cache memory 1215, andreturns the processing to the control CPU 1201. The processing of theface detection processing unit 1213 is thereby finished.

FIG. 19A depicts one example of the image file. FIG. 19B depicts oneexample of a region-of-interest determination processing based on a facearea determination result of the face detection processing.

The image file in which person images are present as shown in FIG. 19Ais subjected to the face detection processing by the face detectionprocessing unit 1213 by the processing flow described with reference toFIG. 18. As shown in FIG. 19B, areas determined as face areas areportions indicated by 1303 and 1305. As shown in FIG. 19B, the faceareas are recognized as circular graphic data.

The face areas detected by the face detection processing are determinedas regions of interest and applied to control over display of list ofimage files while overlapping them with one another. objects to theentire image data is high, the face detection processing can beperformed with high detection efficiency. If the occupying ratio is low,the face detection processing unit 1213 cannot often detect the faceareas. A processing in this case will be described with reference toFIGS. 20A and 20B.

FIG. 20A depicts one example of the image data. FIG. 20B depicts oneexample of the region-of-interest determination processing based onpicture-taking information.

The image file photographed by the digital camera or the like can holdinformation during photographing (Exif information) therein. This Exifinformation includes position information during photographing.According to this embodiment of the present invention, therefore, if theface detection processing fails, focal position information included inthe Exif information is read. The focal position information correspondsto ranges indicated by reference symbols 1403 and 1405 in FIG. 20A.

Based on this focal position information, a range indicated by a circleand including the position information is recognized as the region ofinterest. The region of interest corresponds to a range indicated byreference symbol 1407 in FIG. 20B.

In this way, if the face areas cannot be detected from the target imagefile, the region of interest is determined based on the focal positioninformation included in the Exif information.

FIG. 21 is a flowchart which depicts a processing flow for theregion-of-interest determination processing.

This processing is executed before the control over display of a list ofimage files while overlapping the image files with one another. When theprocessing starts, a step S1201 is executed.

In the step S1201, the image files are read from the image storage unit1205, the memory card 1207, and the external input terminal 1209,decoded by the image decoder unit 211, and stored in the cache memory215. Thereafter, a step 1203 is executed.

In the step S1203, the face detection processing performed by the facedetection processing unit 1213 described with reference to FIG. 18 iscalled for the read image files. Thereafter, a step S1205 is executed.

In the step S1205, the processing is branched depending on whether aface area is present based on the face detection processing result. Ifthe face area is present, the processing goes to a step S1207. If noface area is present, the processing goes to a step S1209.

In the step S1207, face area data stored in the cache memory isregistered as region-of-interest data on the target image file since itis determined that the face region is present in the target image file.After the registration, the processing is finished.

In the step S1209, the processing is branched depending on whether theExif information is included in the target image file. If the Exifinformation is present, the processing goes to a step S1211. If the Exifinformation is not present, the processing goes to a step S1215.

In the step S1211, the Exif information is read from the target imagefile, and the focal position information is further read from the Exifinformation. Thereafter, a step S1213 is executed.

In the step S1213, a processing for converting the focal positioninformation described with reference to FIGS. 20A and 20B into a focalposition region is executed. This focal position region is registered asthe region of interest. After the registration, this processing isfinished.

In the step S1215, a central position of the image file is registered asthe region of interest since no face area is present in the target imagefile and no Exif information can be acquired from the target image file.After the registration, the processing is finished.

The processing described above is executed to all image files to bedisplayed as a list while being overlapped with one another.

FIG. 22A depicts one example in which image files are displayed whilebeing overlapped with one another at random. FIG. 22B depicts oneexample in which image files are displayed while being overlapped withone another in consideration of magnitudes of regions of interest.

In display of a list of a plurality of image files while overlappingthem with one another, regions of interest 1701, 1703, and 1705 ofrespective image files 1501, 1503, and 1505 are always displayed asprotection regions on the monitor screen. This enables improving usersearch performance. Further, a region other than the region of interestof each image file is arranged, as a region of no interest, to be hiddenby the other image file, thereby making it possible to efficientlydisplay the image files.

As compared with the instance of displaying the image files whileoverlapping them with one another at random as shown in FIG. 22A, themonitor screen can be effectively used when the image files aredisplayed while being overlapped with one another so that the image filehaving a wider region of interest is put on the image files havingsmaller regions of interest as shown in FIG. 22B. By introducing thisprocessing, it is possible to display the image files more efficiently.

FIG. 23 is a flowchart which depicts a processing flow for an overlapdisplay processing.

This processing is executed when the user selects display of a list ofimage files while overlapping them with one another. When the processingstarts, a step S1301 is executed.

In the step S1301, the region-of-interest determination processingdescribed with reference to FIG. 21 is called. This region-of-interestdetermination processing is executed for all display target image files.Thereafter, a step S1303 is executed.

In the step S1303, in response to the region-of-interest determinationprocessing in the step S1301, a processing for rearranging the imagefiles in a descending order of magnitudes of the regions of interest ofthe respective image files is executed. Thereafter, a step S1305 isexecuted.

In the step S1305, a variable N that indicates an image file is set at 1that indicates a leading image file. This signifies that as N is larger,the image file is displayed while being overlapped with the other imagefiles at a lower position. Thereafter, a step S1307 is executed.

In the step S1307, a processing for determining the position of theimage file indicated by the variable N is executed. To determine theposition of the image file, coordinates of the position are obtained byrandom numbers. In the determination of the position of the image file,the coordinates thereof are selected so that the region of interest isexposed as much as possible and so that the region of no interest ishidden by the higher image files as much as possible. Thereafter, a stepS1309 is executed.

In the step S1309, it is determined whether the arranged image filefalls within a restricted angle range. If an image display angle isexcessively wide, the user feels the impression that the image isdifficult to view. If it is determined that the image file is out of therestricted angle range, the processing returns to the step S1307. If itis determined that the image file falls within the restricted anglerange, the processing goes to a step S1311.

In the step S1311, it is determined whether a higher image file isoverlapped with the region of interest of the arranged target imagefile. If it is determined that the higher image file is overlapped withthe region of interest of the target image file, the processing returnsto the step S1307. If it is determined that the higher image file is notoverlapped with the region of interest of the target image file, theprocessing goes to a step S1313.

In the step S1313, the processing is branched depending on whether thevariable N indicates a final image file of the display target imagefiles. Namely, if the image file is the final image file, the processinggoes to a step S1317. If the image file is not the final image, theprocessing goes to a step S1315.

In the step S1315, the variable N is incremented by one, whereby theprocessing for determining the arrangement position of the next imagefile is executed.

In the step S1317, since the arrangement positions of all the displaytarget image files are determined, a processing for allowing the imagedisplay unit 1201 to display a list of image files while overlappingthem with one another is called. After displaying the list, theprocessing is finished.

FIG. 24 depicts one example of displaying the list of image files whileoverlapping the image files with one another. As is obvious from FIG.24, all the regions of interest of the respective image files appear onthe image display unit 1201 without being hidden.

(Fifth Embodiment)

An image display apparatus according to a fifth embodiment of thepresent invention is basically equal in configuration to that accordingto the fourth embodiment except that the focal position information haspreference over the face detection processing in the region-of-interestdetermination processing described with reference to FIG. 21.

FIG. 25 is a flowchart which depicts a processing flow for theregion-of-interest determination processing according to thisembodiment.

This processing is executed before the control over display of a list ofimage files while overlapping the image files with one another. When theprocessing starts, a step S1401 is executed.

In the step S1401, the image files are read from the image storage unit1205, the memory card 1207, and the external input terminal 1209,decoded by the image decoder unit 211, and stored in the cache memory215. Thereafter, a step 1403 is executed.

In the step S1403, the processing is branched depending on whether theExif information is included in the target image file. If the Exifinformation is present, the processing goes to a step S1407. If no Exifinformation is present, the processing goes to a step S1405.

In the step S1405, the face detection processing performed by the facedetection processing unit 1203 and described with reference to FIG. 18is called for the read image file. Thereafter, the processing goes to astep S1411.

In the step S1407, the Exif information is read from the target imagefile, and focal position information is further read from the Exifinformation. Thereafter, a step S1409 is executed.

In the step S1409, the processing for converting the focal positioninformation described with reference to FIGS. 20A and 20B into a focalposition region is executed. This focal position region is registered asthe region of interest. After the registration, this processing isfinished.

In the step S1411, the processing is branched depending on whether theface area is present based on the face detection processing result. Ifthe face area is present, the processing goes to a step S1413. If theface area is not present, the processing goes to a step S1415.

In the step S1413, face area data stored in the cache memory isregistered as region-of-interest data on the target image file since itis determined that the face area is present in the target image file.After the registration, this processing is finished.

In the step S1415, a central position of the image file is registered asthe region of interest since no face area is present in the target imagefile and no Exif information can be acquired from the target image file.After the registration, the processing is finished.

(Sixth Embodiment)

An image display apparatus according to a sixth embodiment is basicallyequal in configuration to that according to the fourth embodiment exceptthat the region of interest is a logical OR between a face detectionregion and picture-taking information.

FIG. 26 depicts a result of detecting the region of interest. In FIG.26, a region 1501 is a region of interest obtained by the face detectionprocessing, a region 1503 is a focal region, and a region 1505 is aregion of interest converted from the focal region 1503. In thisembodiment, the region of interest is a logical OR between the region1501 and the region 1505.

(Seventh Embodiment)

An image display apparatus according to a seventh embodiment isbasically equal in configuration to that according to the secondembodiment except that the operation flow of display control unit 110differs. FIG. 27 is a flowchart which depicts the processing flow of thedisplay control unit 110 according to this embodiment. Types of anexposure control mask employed in the processing shown in FIG. 27 areequal to those shown in FIG. 8. Exposure control masks different inexposure amount and exposure position are prepared according to focalpositions or photographic scene modes. The processing operations of thedisplay control unit 110 will be described with reference to FIGS. 27and 8A to 8C.

The display control unit 110 acquires the total number of image filesrecorded in the memory card (in a step S1501), and executes a facedetection processing to first image data (in steps S1503 and S1505).

If a face is detected in the target image file (“Yes” in a step S1507),the display control unit 110 selects an exposure control mask having afocal position corresponding to the focal position information fromamong three masks shown in FIG. 8A, i.e., the left exposure mask, thecenter exposure mask, and the right exposure mask (in a step S1509).

If no face is detected in the target image file (“No” in the stepS1507), the display control unit 110 determines whether focal positioninformation is present in the target image file (in a step S1513). Ifthe focal position information is present, the display control unit 110selects one of the exposure control masks shown in FIG. 8A (in a stepS1515).

If no focal position information is present (“No” in the step S1513),the display control unit 110 reads Exif information and determineswhether photographic scene information is present (in a step S1517). Ifthe photographic scene information is present (“Yes” in the step S1517),the display. control unit 110 selects an exposure control maskcorresponding to the photographic scene mode information (in a stepS1519).

For subsequent image files, the display control unit 110 carries out thesame processings to determine the exposure control masks for therespective image data (in steps S1523, S1505, and the following).

What is claimed is:
 1. An image display method of displaying a pluralityof images with overlap on a screen, the method comprising the steps of:seeking to acquire focal position information of each of the pluralityof images, wherein the focal position information indicates a focalposition of the image corresponding to a point on which a camera wasfocused during photographing and which is recorded with the image by thecamera; registering a region of interest of each image; determining adisplay position on the screen of each of the plurality of images basedon the registered region of interest of the image, such that the regionof interest of the image is not hidden by other images; and displayingthe plurality of images with overlap on the screen in accordance withthe determined display positions, wherein, in a case where focalposition information is acquired from an image in said seeking step, apartial area of the image including a focal position indicated by thefocal position information is registered as the region of interest ofthe image, and in a case where focal position information is notacquired from an image in said seeking step, a partial area of the imageincluding a face area of an object contained in the image is registeredas the region of interest of the image.
 2. An image display methodaccording to claim 1, wherein said determining step includes determiningthe order of superposing the plurality of images such that the pluralityof images are superposed in a descending order of magnitudes of theregion of interest.
 3. A non-transitory computer-readable storage mediumstoring a computer-executable program for implementing a methodaccording to claim
 1. 4. An image display method according to claim 1,wherein in a case where no face area of any object is contained in theimage, a partial area of the image including a central position of theimage is registered as the region of interest of the image.
 5. An imagedisplay method according to claim 1, wherein the face area of the objectcontained in the image is specified by automatic face detection.
 6. Animage display apparatus for displaying a plurality of images withoverlap on a screen, comprising: an acquiring unit that seeks to acquirefocal position information of each of the plurality of images, whereinthe focal position information indicates a focal position of the imagecorresponding to a point on which a camera was focused duringphotographing and which is recorded with the image by the camera; aregistering unit that registers a region of interest of each image; adetermination unit that determines a display position on the screen ofeach of the plurality of images based on the registered region ofinterest of the image, such that the region of interest of each image isnot hidden by other images; and a displaying unit that displays theplurality of images with overlap on the screen in accordance with thedetermined display positions, wherein, in a case where said acquiringunit acquires focal position information from an image, said registeringunit registers a partial area of the image including a focal positionindicated by the focal position information as the region of interest ofthe image, and in a case where said acquiring unit does not acquirefocal position information from an image, said registering unitregisters a partial area of the image including a face area of an objectcontained in the image as the region of interest of the image.
 7. Animage display apparatus according to claim 6, wherein said determinationunit further determines the order of superposing the plurality of imagessuch that the plurality of images are superposed in a descending orderof magnitudes of the region of interest.
 8. An image display apparatusaccording to claim 6, wherein in a case where no face area of any objectis contained in the image, said registering unit registers a partialarea of the image including a central position of the image as theregion of interest of the image.
 9. An image display apparatus accordingto claim 6, wherein the face area of the object contained in the imageis specified by automatic face detection.